SPIN GLASSES AND STEIN'S METHOD 



SOURAV CHATTERJEE 

Abstract. We introduce some applications of Stein's method in the 
high temperature analysis of spin glasses. Stein's method allows the 
direct analysis of the Gibbs measure without having to create a cavity. 
Another advantage is that it gives limit theorems with total variation 
error bounds, although the bounds can be suboptimal. A surprising 
byproduct of our analysis is a relatively transparent explanation of the 
Thouless- Anderson-Palmer system of equations. Along the way, we de- 
velop Stein's method for mixtures of two Gaussian densities. 



1. Introduction and results 

1.1. The Sherrington-Kirkpatrick model. Let N be a positive integer 
and let S^r = {—1,1}^. A typical element of a = (a\, . . . , <jn) £ is 
called a 'configuration' of spins. Let g = (gij)i<i<j<N be a collection of 
independent standard Gaussian random variables, called the 'disorder' in 
our context. Given a realization of g, fix any f3 > and h £ K and define a 
probability distribution Gn on S^r as 

G N (a) = Z' 1 exp(^= ^ 9ij^j + h ^ *X 

Here Zn = Z]y((3,h) is the normalizing constant (partition function), and 
Gn is the 'Gibbs measure'. What we have just defined is the well-known 
Sherrington-Kirkpatrick (SK) model of spin glasses [26]. The parameter f3 
is called the 'inverse temperature' of the model, and h is called the 'external 
magnetic field', following the conventions of Talagrand |30j. Physicists would 
replace the h with /3h, but the definitions are mathematically equivalent. 

Configurations chosen independently from the Gibbs measure (i.e., given 
the disorder) are denoted by a 1 , a 2 , etc. These are called 'replicas' in physics. 
If a 1 , . . . , a k are replicas and / is a function on S^, then as usual we define 

(f(a\...,a k )):= £ f(a\...,a k )G N (a 1 )---G N (a k ). 
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In the terminology of disordered systems, (/(c)) is known as the quenched 
average of f(cr). 

The 'overlap' between a pair of replicas a 1 and a 2 , chosen independently 
from the Gibbs measure, is defined as 

1 N 

i=i 

The 'high temperature phase' of the SK model corresponds to the set of 
(/3, h) for which there is a number q = q((3, h) < 1 such that the overlap R12 
is approximately equal to q with high probability under the Gibbs measure. 
This can be made precise in various ways, and the form that will be most 
suitable for us in this article is: 

(2) E((R 12 -q)*)<^±, 

where C((3,h) is a constant that depends only on (3 and h. (At this point, 
let us declare that throughout this paper, statements like "T < C(/3,h) n 
stands for "the term T can be bounded by a constant that depends only on 
(5 and h" .) It is known (see e.g. [30], p. 72) that the constant q must satisfy 

(3) q = Etanh 2 (pz^/q + h), 

where z is a standard Gaussian random variable. 

It is not very difficult to show that a consequence of the concentration of 
the overlap is that small collections of spins become approximately indepen- 
dent under the Gibbs measure (see [30], Theorem 2.4.10). However, they 
are not identically distributed unless h = 0. One important objective of the 
theory of spin glasses is to find ways to compute the marginal distributions 
of the spins. A way to do this is via the Thouless-Anderson-Palmer (TAP) 
equations, which we study later in this article. 

The high temperature phase of the SK model under zero external field 
was studied rigorously by Aizenman, Lebowitz, and Ruelle pQ. A more 
systematic and powerful approach via stochastic calculus was developed by 
Comets and Neveu |14] and extended by Tindel [33]. The high temperature 
phase for h 7^ was rigorously investigated by Frohlich and Zegarlihski [15] 
and more extensively by Shcherbina [25] and Talagrand [29]. An extremely 
thorough rigorous treatment of the high temperature phase with many new 
results appeared in Chapter 2 of Talagrand's book [30J. An important result, 
shown in [30j, Theorem 2.5.1, is that there exists a constant 0q > such 
that ([2]) holds whenever (3 < (3q. In this manuscript, this is only result we 
borrow from the existing theory of spin glasses. We did not attempt to prove 
this via Stein's method. 

As of now, even the low temperature phase is somewhat mathemati- 
cally tractable, following the deep contributions of Guerra |19j . Guerra and 
Toninelli [18], Talagrand [31] . and Panchenko [22j . The recent paper of 
Comets, Guerra, and Toninelli [13] connecting the SK model and its lattice 
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counterpart is also of interest. For a review of the extensive but mostly 
unrigorous developments in the theoretical physics literature, let us refer to 
the classic text of Mezard, Parisi, and Virasoro |21j . 

1.2. The TAP equations. Since the spins can take only two values, the 
quenched distribution of the spin at site i is completely described by the 
quantity (<7j). One of the main approaches (as outlined in [21] ) to under- 
standing the high temperature phase of the SK model is to understand the 
quantities (o"i), • • • , (o"jv) via the Thouless- Anderson-Palmer system of equa- 
tions: 



(4) (o-i) «tanh (-^=Y,9ijM + h -l3 2 0--Q)(^) > i = h---,N. 



Here ~ means, vaguely, 'approximately equal with high probability'. Physi- 
cists usually write exact equalities in such cases. 

This self-consistent system of equations has a unique solution with high 
probability if (3 is small. It was physically argued by Thouless, Anderson, 
and Palmer [32] that the quantities (<7i), . . . , (ctjv) must satisfy these equa- 
tions 'in the large N limit' at any temperature and external field. The first 
rigorous proof of the validity of the TAP equations in the high temperature 
phase (where ([2]) holds) appeared twenty-six years after the publication of 
the physics paper, in Talagrand's book ([30], Theorem 2.4.20). However, 
Talagrand's theorem in [30] does not show that all N equations hold si- 
multaneously with high probability. This has been proved more recently 
(Talagrand, private communication), and is going to appear in the forth- 
coming edition of [30] . 

Talagrand's proof is based on a remarkable rigorous formulation of the 
cavity method, which involves studying the system after 'removing the last 
spin'. This procedure is known as 'creating a cavity'. Now, if one wants to 
study the Gibbs measure directly, without having to resort to the essentially 
inductive process of creating a cavity, is there a way to proceed? This is a 
key focus in this paper. Let us begin by outlining the approach for the case 
of the TAP equations and understanding how they arise. 

1.3. The Onsager correction term. The first step is to observe that 
the conditional expectation of Oi given under the Gibbs measure is 

simply tanh(/3£j + h), where li is the local field at site i, defined as 



The proof of this is quite trivial, following directly from the form of the 
Gibbs measure. It follows that 





(6) 



(o-i) = (tanh(/% +h)), i = 1, . . • , N. 
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Thus, if we could understand the distribution of the local fields under the 
Gibbs measure, the problem of computing (o"j) would be solved. This moti- 
vates the study of the limiting behavior of the local fields. 

Incidentally, the naive mean field heuristic would dictate that the 'average 
can be moved inside the tanh', and 

(ai) = (tanh((3£i + h)) tanh + h) 

= tanh S 9ij + hj ■ 

However, the naive heuristic does not work for the SK model. This is not 
surprising since the local fields are unlikely to be concentrated. The famous 
discovery of Thouless, Anderson, and Palmer [32] is that the average can 
still be moved inside the tanh, but only after adding what has come to be 
known as the Onsager correction term, — /3 2 (1 — 9)(<XjY As stated before, 
this gives the TAP equations 



(ffi) wtanhj — 2^913(03) + h-/3 A (l-q){o-i) ), i = l,...,N. 

In view of the approximate independence of the spins under the Gibbs mea- 
sure at high temperature, it seems natural to surmise that the local fields 
would be approximately Gaussian. Surprisingly, this is not the case. Rather, 
the explanation for the Onsager correction is hidden in a property of convex 
combinations of pairs of Gaussian distributions. 

1.4. Onsager correction and mixture Gaussians. For any \i G R, a > 

0, let N(fj,,o~ 2 ) denote the Gaussian distribution with mean \x and variance 
a 2 , with density function 

(7) ^(x) = -^e-(^) 2 /^ 2 . 

V lira 



The mixture (i.e. convex combination) of two Gaussian densities has a cu- 
rious connection with the tanh function. Suppose X is a random variable 
following the mixture density p<^ AtliCT 2 + (1 — p)4> a v ■ Suppose fj,\ > fi2, and 
let 

(8) a = ^^' 6= 2 log T^-^^- 

Then a simple computation gives 

(9) Etanh(aX + b) = tanh(aE(X) + b-(2p- l)aV). 

That is, the 'expectation can be moved inside the tanh', after incurring 
the quadratic 'correction term' —(2p— l)a 2 a 2 . (The proof of this identity is 
sketched in the Subsection ll.81 With other values of a and b, the expectation 
can still be moved inside, but the correction term will no longer have a simple 
form.) The similarity with the Onsager correction is more than superficial: 
in fact, it turns out that the distribution of the local fields under the Gibbs 
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measure can be approximated by a mixture of two Gaussian densities, and 
the correction term in the above equation indeed corresponds to the Onsager 
correction term in the TAP equations. 

1.5. Limit law for the local fields. The precise result about the limiting 
distribution of the local fields can be described in the following way. For 
each 1 < % < N, let 

(10) n = nig) := -j=Y^9ij(<rj) - 0(1 - q)(<ri), 

p Pn+h 

(11) Pi 



e 0ri+h _|_ e -f3ri-h ' 

and let z/j be the random probability measure on R with the mixture Gauss- 
ian density 

Pi<f > r i +0(l-q),l-q + (1 ~ Pi)<Pn-0(l-q),l-q- 

Then the distribution of the local field ii (defined in ([5])) under the Gibbs 
measure is close to z/j, in the sense that the difference between the two 
(random) measures converges in probability to the zero measure. A more 
quantitative result is as follows. 

Theorem 1.1. Suppose (3 and h are such that ([2]) is satisfied for some 
q < 1. Let vi, . . . , i/jv be defined as above. Then for any bounded measurable 
u : R — > M and any 1 < i < N , we have 



H W))- / u(x)ui(dx) < 



2 ntR f.ML.112 



c{p,h)\H« 



<N 

where C(/3,h) is a constant depending only on (5 and h. 

Taking u(x) = tanh(f3x + h) and using the connection ([9]) between mixture 
Gaussian distributions and the Onsager correction, we can now readily prove 
the TAP equations. 



Corollary 1.2. If is satisfied, then for each 1 < i < N we have 
E^o-i) - tanh(-^L X>*Ai> + h ~ /? 2 (1 " «)<^> 



N 



Proof. By ([6]), we know that (c7j) = (tanh(/3-£j + /i)). Now, for the mixture 
Gaussian density Vi, a simple computation shows that a = (3 and b = h, 
where a and b are defined as in ([8]). Taking u(x) = tanh(/?:r + /i) in Theo- 
rem [TTT] and using the property ([9]) of mixture Gaussian densities, it is not 
difficult to verify that we get the stated result. □ 

Note that Talagrand's version of Corollary 11.21 (|30j, Theorem 2.4.20) has 
an error bound of order 1/N, and so our result is suboptimal. However, we 
are not entirely certain whether Theorem 11.11 itself is suboptimal (although 
it probably is), because improving the l/y/N bound in the proof seems 
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to require some kind of smoothness for the function u, which we are not 
assuming. 

1.6. An explanation of the mixture Gaussianity. Since we know the 
conditional distribution of o"j given it, and we know the marginal laws of 
ii and o"j via Theorem 11.11 and Corollary 11.21 it is possible to compute the 
conditional law of ii given a% by Bayes' rule. It turns out that given Oj = 1, 
the law of li is approximately Gaussian with mean rj + [3(1 — q) and variance 
1 — q, and given U{ = —1, the law of £j is approximately Gaussian with mean 
ri — (3(1 — q) and variance 1 — q. Thus, the marginal distribution of 
under the Gibbs measure is approximately a convex combination of these 
two distributions. 

1.7. Stein's method. We prove Theorem 11.11 using our version of the clas- 
sical probabilistic tool developed by C. Stein [271 [28]. Incidentally, it is also 
possible to prove it using standard techniques from the cavity method as 
developed by Talagrand. However, as we shall see below, one advantage of 
Stein's method, besides the total variation error bounds, is that it allows 
us to 'discover' the result before proving it. Let us give a brief primer on 
Stein's method below. 

Suppose we want to show that a random variable X has approximately 
the same distribution as some other random variable Z. The basic idea 
behind Stein's method of distributional approximation 27, 28J is as follows. 

1. Identify a "Stein characterizing operator" T for Z, which has the 
defining property that for any function / belonging to a fixed large 
class of functions, WTf(Z) = 0. For instance, if Z is a standard 
Gaussian random variable, then Tf(x) := f'(x) — xf(x) is a charac- 
terizing operator, acting on all locally absolutely continuous /. 

2. Take a function u and find / such that Tf(x) = u(x)— 1 &u(Z). Relate 
the smoothness properties of / to those of u. 

3. By the definition of / it follows that \Eu(X) -¥,u(Z)\ = |E(T/(X))|. 
Compute a bound on \M(Tf(X))\ by whatever means possible. 

The procedure for normal approximation can be simply described as follows: 
if we want to show that a random variable X is approximately standard 
Gaussian, Stein's method demands that we show M(f'(X) —Xf(X)) ~ for 
every / belonging to a large class of functions. 

Although the raw version of the method as stated above may seem like a 
trivial reduction, the replacement of u(x) — 'Eu(Z) by Tf(x) often gives a 
high degree of maneuverability in practice. While steps 1 and 2 have to be 
carried out exactly once for every distribution of Z, the execution of step 3 
depends heavily on the problem at hand. A number of techniques for carry- 
ing out this step are available in the literature, e.g. exchangeable pairs [28] . 
diffusion generators [5], dependency graphs [HE], size bias couplings [T7j . 
zero bias couplings [16], couplings for Poisson approximation [I2j [6], spe- 
cialized procedures like [23j [24l EQ] , and some recent advances [U (H [TUl E] • 
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Incidentally, Stein's method was applied to solve a problem in the interface 
of statistics and spin glasses in [7]. 

1.8. Stein's method for mixture Gaussians. For any a,b,fx € R and 
a 2 > 0, let M(a,b, fi,a 2 ) be the probability distribution on R with density 
function 

(12) VW-o 3 = Z a,t,^ cosh («* + bje-^) 2 /^ . 

where the normalizing constant is given by 



^a,fe,/i,cr 2 = v27r<rcosh(a/x + 6)ea a CT . 

A simple verification shows that M(a,b, fj,,a 2 ) is in fact a mixture of two 
Gaussian distributions: 

(13) ll>a,b,tt,a*(B) = P0/*+«r , ,o- a ( aj ) + (1 - p)<l>p-a0*,o* (x), 

where 4> stands for the Gaussian density function (J7J) and 



gCifi+b _|_ g—afi—b' 

Thus, the distributions representable as M(a,b, fi, a 2 ) are exactly the dis- 
tributions arising as mixtures of two Gaussian densities. Note that in par- 
ticular, M(0, 0, fi, a 2 ) is just the Gaussian distribution with mean /j, and 
variance a 2 . An interesting fact about this class of distributions, required 
for the proof of Corollary 11.21 is that 

(14) / tanh(ax + b)^ ab4Ji(T 2(x)dx = tanh(a/x + b). 

The computation can be easily done using the convenient representation (|12|) . 
Note that this is exactly the relation ([9]). Again, using (I12p it is not difficult 
to verify that the operator 

Tf(x) = f'(x) - f^2^ - atanh(ax + b)\ f(x) 

is a Stein characterizing operator for M(a,b, fi, a 2 ). Roughly, this means 
that to show that a random variable W approximately follows the distribu- 
tion M(a,b, (i, a 2 ), we have to show that for all /, 

E (f'(W) - ~ a tanh(aW + 6) J f(W) \ « 0. 

To develop Stein's method for this class of distributions, we have to solve 

(15) Tf(x)=u(x)- / u(t)ip aAfM}(7 2dt 



for arbitrary u : R — > R, and relate bounds on / and its derivatives to 
properties of u. 

Now note that by (113D . the measure Vi in Theorem ll.il can be alternatively 
written as M(/3, h,ri, 1 — q). The randomness of rj adds an extra level of 
complexity: We also have to analyze the dependence of the function / in (fl~5j) 
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on the parameter \x. So we should start with /(x,/x) rather than f(x). The 
bounds required for Stein's method are summarized in the following lemma. 

Lemma 1.3. Fix a, b € R and a 2 > 0, and a bounded measurable function 
u : M — > R. Then there exists an absolutely continuous function f : M 2 —* R 
smc/i i/iai /or a// 



(x, fj) — ( — 2 atanh(ax + b) ] /(x, fj,) 



= u{x)- / u(t)i> ab „ ff 2(t)dt. 
Jr 

Moreover, we can find a solution f such that for some constant C(a, o~) 
depending only on a and a we have that \f\, \^\, and |^| are all uniformly 
bounded by C(a,cr)\\u\\co- 

Note that the case a = b = covers the case of the pure Gaussian distri- 
butions. The proof of this lemma, which is quite elementary but tedious, is 
relegated to the end of the manuscript. 

1.9. How to apply Stein's method. To see how Stein's method can be 
used in the SK model, let us sketch a very simple example: the unconditional 
(i.e. average over the disorder) distribution of the local field at site 1 when 
(3 < 1 and h = 0. This is a special case of Theorem 11.11 Since R12 
concentrates around zero in this regime (see e.g. [I], or [30], Chapter 2), we 
have q = q((3,0) = 0. Also, by symmetry, (o~j) = for each i. Therefore 
the measure is actually a nonrandom probability measure, namely, the 
mixture Gaussian density h<t*j3,i + w-/9,i- Clearly, the nonrandomness of the 
limiting distribution hugely simplifies our goal. Let us now see how we can 
prove via Stein's method that this is the limiting distribution of the local 
fields. 

Recall that l\ = -7= Ylj=2 9 1 3 cr 3- ^ x a smo °th function /. For each 
j = 2,..., N, let 



N 



Then we have 
(16) 5>iA' = <Wi)>. 



On the other hand, an easy computation gives 

dhj = (f'(ii)) + P(c-if(h)) - f3(o-if(h))(o-iQ'i) 
dgij N 

Note that i\ does not depend on a%, and the conditional expectation of a\ 
given o"2, . . . , o~n is tanh(/3£i). Thus, 

(o-if(h)) = (tanh(/?4)m)>. 
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Again, it follows from the high temperature condition ([2]) for f3 < 1 and 
h = that for 2<j<N, 

(aiaj) « (oi)(o}) =0. 

Combining, we see that 

N ah- 

(17) Yl of. ra </'&)> + ^(tanh^)/^)). 

Now, using integration by parts for Gaussian random variables, we get 

In view of (|16jl and (|17|) . this is equivalent to 

(18) E<4/(4) - - /3tanh(/34)/(4)> « 0. 
As noted in Subsection 11.81 

Tf(x) = xf(x) - f\x) - /3tanh(/3x)/(x) 

is a Stein characterizing operator for the mixture Gaussian density 5^,1 + 
2^-/3,1. Note that this procedure 'discovers' that the (averaged) limiting 
distribution of l\ is the above Gaussian mixture. 

1.10. Quenched distributions and the Approximation Lemma. In 

the above example, we sketched a derivation of the limiting unconditional 
(i.e. averaged over disorder) distribution for the local field at site 1, es- 
sentially using Gaussian integration by parts. To prove the result for the 
quenched distribution, it does not suffice to show (fTHj) . but rather, we have 
to show 

(hf(£i) - f'(£i) - f3tanh((3h)f(h)) ~ with high probability. 
In other words, we have to show 

N N dh- 

Y^gijhj ~ o 3 with high probability. 

j=2 j=2 9lj 

This is a recurring issue whenever we have to prove a quenched CLT. The 
following result, which we call the 'approximation lemma', becomes our main 
tool. The proof of the lemma is so short that we present it right away. 

Lemma 1.4. Suppose g = (gi, . . . ,g n ) is a collection of independent stan- 
dard Gaussian random variables, and h\,...,h n are absolutely continuous 
functions of g. Assume that hi are elements of the Sobolev space H > 2 with 
respect to the Gaussian measure on R. Then 
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Proof. By taking convolutions with smooth kernels, we can assume that 
hi, . . . , h n are twice continuously differ entiable. Let 



n , 
.7=1 V 



9hj 



Then 



Ehi = ±E(g 3 hjh-^Lhy 



Integration-by-parts gives 



. _ . , ... , dhj , , dh 

Hg 3 h 3 h)=E[ J J-h + h ] - 



Thus, 



Now 



Therefore, 



Eh 2 = 

j 



/ dh 



sr^f dhk dh 



r \ / I 



d 9j f^V <% dgjdg k 



Eh 2 = 

j=i j,k=i 
Again, using integration-by-parts, we see that 

e( h- ^ k \ e( ^ j ^ k + h ^ 
V k 3 d 9j J \dg k dgj 3 dgjdg k J ' 

This completes the proof. □ 

1.11. Other results. The following theorems are some further examples of 
CLTs for the SK model that can be proved via Stein's method. In all cases, 
we obtain total variation error bounds. Although the bounds are probably 
suboptimal, this is the only method available that can give such bounds. 



1.11.1. The cavity field. Suppose g\, . . . , gjy are i.i.d. standard Gaussian ran- 
dom variables, independent of the disorder {gij)i<j<N- The 'cavity field' t 
is defined as 
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The name 'cavity field' comes from the role played by £ in the cavity method 
for solving the SK model in the high temperature regime. Note that the 
quenched average of t is 

1 N 

<<> = 7^ 

v 1=1 

The following result states that under the Gibbs measure, I is approximately 
Gaussian with mean (£} and variance 1 — q. The original proof of this result, 
without the error bound, can be found in Talagrand [30J, page 87. 

Theorem 1.5. Suppose the high temperature condition ([2]) is satisfied and 
u : R — » R is a bounded measurable function. Then 

2 nia mil.. 112 

E (<«(*))- / u{t)<j) m _ q {t)dt 



, C(P,h)\\u\ 



oo 



'N 

where (j) is the Gaussian density defined in ([7]). 

1.11.2. The Hamiltonian. Our next limit theorem is about the quantity 



1 v-^ VN8 
(20) H := — 2^ 9ijO-iO-j — . 

i<j<N 

Note that this is just a linear transformation of the interaction term in the 
hamiltonian. We show that in the regime (3 < 1, h = 0, the quenched law 
of H is asymptotically Gaussian with mean and variance 1/2. The case 
of general (3 and h, even in the high temperature phase, seems to be much 
harder, and is currently under investigation. 

Theorem 1.6. Let H be defined as above. Suppose j3 < 1, h = 0, and 

u : R — > R is a bounded measurable function. Then 



E (u(H))- / u(t)(f> 0jl/2 (t)dt 



2 /-vmn„,ii2 



< 



cmu\ 

N 



T/iis <7i?;es a total variation error bound of order 1 / v iV in £/ie central limit 
theorem for H . 

Again, this was originally proved in Comets and Neveu |14j . Proposition 5.2, 
albeit without an error bound. 

1.11.3. Quenched average of the spin at a site. It is natural to ask about 
the limiting distribution of the random variables ((o"j))i<j<jv. Although the 
joint distribution has no simple description, Talagrand proved that for any 
fixed k, the collection ((<7i))i<i<fc converges in law to (tanh(/?Zjy / g+/i))i<j<fc, 
where z\ , . . . , Zk are independent standard Gaussian random variables (The- 
orem 2.4.12 in [30]). 

Note that the term inside tanh in Corollary 11.21 is simply (3ri + h, with 
T{ defined in (fTUl) . Hence, to compute the asymptotic distribution of (erj), 
it suffices to find out the limit law of r , . The following result shows that r i 
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is asymptotically Gaussian with mean and variance q, and gives a total 
variation error bound. 

Theorem 1.7. Suppose ([2]) holds for some q > 0, and ri is defined as 
in (jlOp , Let z be a standard Gaussian random variable. Then for any 
bounded measurable u : R — > R ; 

|E^)-E^)|<^*ifc. 

AToie i/iai fry Corollary 11.21 i/tis shows that (cj) zs asymptotically distributed 
as tanh.{(3zy/q + h). 

The rest of the paper is organized as follows. Since the complete proofs 
involve some heavy computations, we give brief sketches of the proofs in 
the next section. The details are in Section [3j Section [3j also contains a 
development of Stein's method for mixture Gaussian densities. 



2. Proof outlines 

In this section, we sketch the proofs of the theorem from Section [1] in the 
order of difficulty. 

Sketch of the proof of Theorem 11.51 Recall the definition (jT9j) of the cavity 
field £ and let r = (£). Take any smooth function / : R 2 — > R and for each 
j, let 

Then 

JV 

'£g j h j = ((£-r)f(£,r)). 

3=1 

A careful calculation shows that under (|2|), the approximation lemma can 
be applied, and 

Combining, we get that for any smooth / : R 2 — > R, 

This shows that the law of I under the Gibbs measure approximately satisfies 
the characterizing equation for the Gaussian law with mean r and variance 
1 — q. The proof can now be completed by standard techniques from Stein's 
method. 

Sketch of the proof of Theorem WM Recall the definition (I20j) of the centered 
hamiltonian H, and take any smooth function / : R — ► R. For each i < j < 
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N let 

ft,-, = 

N 



Then 



i<j<N 

In the regime < 1, h = 0, it is known that R\2 = 0(iV _1 / 2 ). Using this 
fact and some calculations, it follows that the approximation lemma can be 
applied to the collection (gjj, ^r,)i<j<7v, and also that 

This shows that for any smooth /, 

(Hf(H) - !/'(#)) ~ 0, 
and Stein's method does the rest. 

Sketch of the proof of Theorem 11.11 Recall the definitions ([5]) and ()10p of 
£j and rj. It suffices to prove the theorem for i = 1. Take any smooth 
/ : R 2 -> E, and for each 2 < j < N, let 

^(5) = -4<(^-<^»/(4,r 1 )). 



Then 

E5lA = <(4-ri)/(4,ri)). 
i=a 

Now /ij depends not only on (gij)2<j<N but also on (gij)2<i<j<N- However, 
we can condition on (gij)2<i<j<N an d then apply the approximation lemma 
to show that 

N N dh- 

In a number of steps, one can show that under ([2]), 

Ej^~(i- 9)(/?tanh(^! + ^)/(£x, n) + ^ (£i,n: 



Combining it follows that for any smooth / : 



p2 



' ] /3tanh(/34 + /i)V(^»n)-|^(^,ri)> wO. 



1 — g J dx 

It turns out that an exact equality in the above equation characterizes the 
distribution v\ from Theorem 11.11 The proof can now be completed by 
Stein's method. 



11 
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Sketch of the proof of Theorem 11.71 Again, it suffices to prove the theorem 
just for r\. By a series of steps involving integration by parts and appli- 
cations of the high temperature condition (|2]), one can show that for any 
smooth / : R — ► R, 

(21) E(ri/(r a ))«gE(/'(ri)<»7i)), 

where 

n N 



Now let 



J=2 



h 3 



Clearly, 

N 

(22) </7i> = 1 + J>A - /3 2 (1 - g)(l - (a,) 2 ). 

3=2 

A series of steps using the high temperature condition ([2]) give 

Applications of fl2J) also imply that the approximation lemma can be used 
in this case to deduce that 

N N dh 

and therefore the last two terms in (|22|) approximately cancel each other 
out, leaving us with the conclusion that (r/i) « 1. Combining with (f2Tj) . we 
see that for any smooth / : R — * R, E(ri/(ri)) ~ gE(/'(ri)). The proof is 
now completed by Stein's method. 

3. Complete proofs 

3.1. Some estimates. Applying Lemma Q3] in our problems require a sub- 
stantial amount of computation. The purpose of this subsection is to orga- 
nize the computations into a friendly and accessible system. 

In this subsection and the rest of the manuscript, we switch to the con- 
vention that C(/3,h) denotes any constant that depends only on (3 and h. 
In particular, the value of C(/3, h) may change from line to line. 

Let us first recall our conventions. Configurations chosen independently 
given the disorder are denoted by a 1 , a 2 , etc. The overlap between a 1 and 
<t 2 is defined as 

1 N 

\ 1 2 

i=l 
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Recall that we have a number q, depending on and h, such that 
(23) E((R 12 -q)^<^l. 

Let us begin our computations with the following straightforward formula: 
For any function v = v(g, a) of the disorder g and the configuration a, and 
any we have 

d(v) I dv \ 



(24) 



I dv \ /( 
\d9ij) + VN {[1 



For each j, let 

(25) aj = aj(g,a) = aj - (aj). 

Then by (121) . 

dg k i dg M ViV ' 
Now let v(g, a) be a bounded function of g and a. Then 

N N 

3=1 3=1 

= E((R 12 - R u - R 23 + R u )v(g, a l )v{g, a 2 )). 

From this and the inequality (|23p . we get 



1 N 

Next, note that 

N ., N 



- C(0,h)(E(v i )) 1/2 



1 v—-\ 1 

((l-^H^a 1 )). 



Thus, we have 



( 1 

.7 = 1 



If w is another function of g and <r, then 
1 N 



N 

3=1 
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Thus, we have 

N \2 r<(R M/"iW„,4\/„„4\\i/2 



\ i=i 7 
Next, note that 

1 N 1 N 

3=1 3=1 

= <(i2i2 - RuHg^^wig, a 2 )). 

This shows that 

1 A w C(/3,/t) (E(u 2 )(ii; 2 )) 1/2 

( 3 °) vE E (MM) ^ 1 v ' ^ 



3=1 



and moreover 

(31) E ( * ^/^A/....AV^ cr 09,fc)(E<« 4 ><« 4 » 1/a 



1 \ 

3=1 7 



The inequality (|30p readily implies the following important lemma. 

Lemma 3.1. Let vi, . . . , ujv, Wi, ■ ■ ■ , wn be arbitrary functions of g and a. 
Then we have 



E 



^ £ <^x^->) < ^E(E<«2><w2» l/a 
i,fc=i 7 fe=i 



The above result is generally used as follows. Given functions /2, • • • , /iv of 
the disorder g, we find t/j and iUj such that 

dfj = (o-jVk) = (wjo-k) 
dgik N N ' 

and apply the bound from Lemma [3.1l to extract information from Lemma [1.4i 
The next lemma is necessary for bounding the moments of functions of 
(g, a) that arise when we try to apply the inequalities derived above. 

Lemma 3.2. Let b\(o~), . . . , b m (a) be arbitrary functions of a, taking values 
in the interval [—1,1]. Then for any k > 1 and any distinct collection of 
indices 2 < ji, . . . , jk < N } we have 

H9ih9ij2 ■ ■ ■ 9ij k {bi){b2) • ■ ■ {bm)) < — ^ , 
where C(m,k) is a constant depending only on m and k. 
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Proof. Let us use induction on k. For k = 1, observe that by integration- 
by-parts and (f24l) . we have 

e(^x^>-"<^>)=e e (^ii< 6 «': 

Z 3 V^f//. _ \TT/, ^ < CM/ 3 



7=£ E <(6«-<6«»^x>II^ 

VJV i=i v v+i 



Now assume that the result is true up to k — 1 (and any to). Again, using 
integration-by-parts and (f24l) . we have 

E (sijiSy 2 • • -9ij k (bi)(b2) • • • (b m )) 

<jL£ E ( 91j ,.. 91 ,£M n fr: 

/3 m / 

= ywY. E ( 91 ^ •••9ij k ((bi ~ < & /»^i^i>II( 6 '' 

ViV 1=1 V 

A straightforward application of the induction hypothesis for k— 1 completes 
the proof. □ 

The following function will appear several times in the sequel. 

1 N 

(32) e 1 = £ 1 ( g ,a) = -=^ f9lj & j . 

Take any k > 1. A simple application of Lemma 13.21 to each term in the 
expansion of ^ shows that 

(33) E(ft)<C(J3,k). 

The important thing is that the bound does not depend on N. 

3.2. Proof of Theorem 11.51 We will continue using the notation intro- 
duced in the previous subsections. Let us briefly recall the setting. Suppose 
<7i,...,<7jv are i.i.d. standard Gaussian random variables, independent of 
(9ij)i<j<N- The cavity field is defined as 

1 N 

Our objective is to show that under the Gibbs measure, £ is approximately 
distributed as a Gaussian random variable with mean 

N 



7^ XX 

and variance 1 — q. 



r := —== > Q-i ( &<i 
N . , 
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Take any bounded measurable function u : R — > [—1,1] and suppose 
/ : R 2 — > R is a solution to 

d f x — y f 

-^-{ x ,y) ~ f(x,y) = u{x) - I u{t)(j)y,i- q {t)dt. 

cx i — q 

For simplicity, we let fa and fa denote ^ and From Lemma 11.31 it 
follows that | f\, \fa\ and I/2I are uniformly bounded by C(/3, h). For each j, 
let 

h r .= ^={a 3 f{i,r)). 



'N 
Then 

N 

(34) £«^ = <(^-r)/(£,r)). 

i=i 

In the rest of the proof, we will simply write /, fa and /2 instead of f(£, r), 
etc. Note that for any j, k, 



dhi I „ x . 1 

Thus, putting 



Vk = <?kfa + fa(ok) and Wj = &jfa + (ajfa), 

we see that 

gfoj _ (<^fc) _ (wj(J k ) 
dg k ~ N ~ N 
Hence by Lemma |3.1| we have 

N 

£ E 

j,k=i 

Again, from (|27p we have 

JV 



dhj dh k \ < C{(3,h) 



dgk dgj J y/N 



2^ C(^,/l) 



< 



5=1 

Combining and applying Lemma 11.41 we get 

.ff l ^*A" C(/3,ft) 



//V 



Again, note that 



7, AT 

j=i ^ i=i 
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By ([28D and (|3U), this gives 




C(0,h) 



N 



Combining ([53)1 . (f35|) and ([36]) . we finally get 



e((u(1)) - [ u{t)4> rtl - q {t)dt 




1 



mi-q)h-{Z-r)ff 



< C(/?,h) 

This completes the proof of Theorem 11.51 

3.3. Proof of Theorem 11.61 Recall that the centered hamiltonian H was 
defined as 



Jul 

Again by Lemma [L3l |/| and |/'| are uniformly bounded by C(P,h). For 
each i,j, let 



In the following, we will write / and /' for f{H{a)) and f'{H{a)) for nota- 
tional convenience. When we have expressions involving multiple replicas, 
/ will stand for flyH^a 1 )). We have 




i<j<N 

Take any u : R — * [— 1, 1], and let / be a solution to 





(37) 




Using identities like 




i,j,k,l 




etc. 



i,j,k,l 



we get 




Similarly, 




2^ ^ C(J3) 
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Another similar verification starting from the formula (I37p shows that 

Kj ■> 

where Jl is a remainder term satisfying 

E(ft 2 ) < 

The proof is now completed by applying Lemma 11.41 

3.4. Proof of Theorem 11.11 It suffices to prove the result for i = 1. Note 
that v\ is simply the probability distribution M((3,h,ri,l — q). Without 
loss of generality, we can assume that ||u||oo < 1- Suppose / : M 2 — » R is a 
solution of the differential equation 

?f(x, y) - - (3 tanh(/3x + h)) f(x, y) 

(38) dx J 

= u(x) - / u(t)lf>p t h,y,l-q(t)dt. 
JR 

By Lemma 11.31 such an / exists and moreover, we can guarantee that |/|, 
and 1 are all bounded by C{(5,h). As before, to lighten notation, 
we let fi and fa denote the two partial derivatives of /. We have to prove 
that for any i, 

2 C(P,h)\\u\\ 2 



where 



E^(u(^)} - J^u{x)ij}^ hinj i^. q dx\ < 



fa) : = -jfi^aijM - Pi 1 - q)(<ri 



oc 



n = ri w , 

By the definition and properties of /, this is clearly equivalent to proving 
that 

E(f 1 (l i ,r i )-( £ -^-/3t a jxh(mi + h))f(£ i ,r i )) 2 < C{ U '> 



1-5 / / VN 

and this is what we aim to prove in the next few pages. Note that it suffices 
to fix i = 1. Recall that we defined 

For each j > 2, let 

hj(9) = i(o-j/(4,n)>, 



where recall that 

N 



. . - 1 ^ 
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and 

1 N 

n =n{g) = - 7 =J2gi j {a j ) - 0(1- q){ai). 

In what follows, the random variable f(£i,ri) is simply denoted by / to 
lighten notation. The distinction between the random variable / and the 
function / should be clear from the context. Similar remarks apply to fx 
and /2 also. 

Now for any j, k > 2, simple applications of (|24p and (|26|) gives 
9% _ 1 /d&j \ 1 / . 0li \ 1 , . , c^n 

+ jj{(?jf)(<n°k - {vi°k))) 

(39) = -l/a^Xf) + Uajha k ) + 4=<*^) ^' 



1A 



A further use of (|24() and (j26|) gives 

9ri <^a> v> v /3 2 (1 - g) 



7^ + Jf2^9u{**Wk) ~ j= {with). 



Recalling the definition (1321) of and putting 

(40) t?i := l + ^KJi -/3 2 (1- ^^cn, 
we see that 

(41) p_ _ W 

%lfc VA 7 

Thus, putting 

v k = -(3cr 1 a k (f) + /icr fe + f 2 (m^k) + (3fcria k - Pf{aia k ), and 
w i = -^j a i{f) + & jh + (vjf2)m + per j fax - 0{&jf)a Xl 
and organizing the terms in (j39H . we get 

dg7k~ N ~ N 

Since /, f\, and fi are uniformly bounded by C(j3,h) and E(^f) < C((3,h) 
by ([33]) , an application of Lemma 13.11 gives 
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Again, since / is bounded by C(f3, h), we can use (|2T|) to get 

N 

(43) ^E(^ 2 



< 

>■ j • — 

3=2 



Applying Lemma 11.41 using the bounds (143)) and (I42|) obtained above, we 
finally get 

(44) V i=2 3=2 j=2 j,k=2 v yiK 

< C(/3,/ i ) 



Note that although hj is a function of the whole collection (gjk)i<j<k<N, 
we can first condition on (gjk)2<j<k<N an d apply Lemma [1.4l and then take 
the unconditional expectation to get the first line in (|44p . 
Now let us define 

7i :=-P<ri(f) + fi + P<rif. 
Then from the expressions (|39() and (j41[) we see that 

N dh- I N f \ 

3=2 ylj 3=2 V 7 



Applying f|28|) for the first term and (|3ip for the second and third terms, 
we have 

<Ef^a-H 2 ^ 

3—2 

Note the most crucial point in the derivation of (|44p and (|45p is that by 
Lemma [L3l the bounds on / and its derivatives depend only on (/?, h). The 
parameter \x in the mixture Gaussian distribution, which equals T\ in this 
proof, does not actually behave as a fixed parameter because we have defined 
/ as a function of two variables, one of which is /j,. This is why we need to 
have / defined on M 2 instead of M 1 . 

Now / = f(£i,ri) does not depend on a\. Also recall that under the 
Gibbs measure, the conditional expectation of o\ given 02, . . . , cjtv is simply 
tanh(/5^i + h). Thus, 

(<7i/> = <tanh(^i + /i)/>. 
Using the above identity to compute (71), we see that 
N 

3=2 

= ({£i-n- (3(1 - q) tanh(/% + h))f) - (1 - q)(h). 
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Combining (j4"l"|) and ((131) . and dividing by 1 — g throughout, we get 



But by the definition ((38]) of f. 

Mh, ri) - (j^J ~ /?tanh(/% + h)j f(£ 1} n) 



= - / ^(^)^/3,h,n,l-g( x ) d2; - 

This completes the proof of Theorem II. 11 



3.5. Proof of Theorem 11.71 Recall the definitions 

1 N 

ri := 7^Z,5ii<^)-^(i-9)<^i) 

V 3=2 

and 

r?i := 1 + - f3 2 (l-q)ai<Ji. 

Let it : R — > [—1, 1] be a measurable map. By Lemma ll.3( there exists an 
absolutely continuous function / : R — > R such that 

/'(x) - ~ /(x) = «(x) - Eu(z^), 

where z is a standard Gaussian random variable, and moreover |/| and \ f'\ 
can be uniformly bounded by C(/3, h). From the definition (jlOp of n, we see 
that 

1 N 

(46) n/(n) = -= ^5ii(^)/(ri) - 0(1 - ?)<<ri>/(ri). 

Now, by integration-by-parts and the identities ((26p and (|41|) . we have 



,'N ^ N 

Thus, we have 

N \ / 

/ I ■ 

E 

WN . , 
(47) J=2 



By ((281) and the bound on |/|, we have 
(48) 
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Similarly, by (|29|) and the bound on \f'\, 
(49) 



C(p,h) 



Combining (06]), (07]), (08]), and (09]), we get 



|E(n/(n)) -#(/' (nXr/i))! < 

V -/V 



Since /' is bounded and 

Eu(n) - En(z^) = g^Efe/Cn) - ri/(n)), 
the proof will be complete if we can show that 

(50) E(( m )-l) 2 < C ^ h) 



'N 

The rest of the proof is devoted to proving (|50|) . From the definition (|4U|) of 
r/i, we get 

a - 

(vi) = i + -j= E aw<w> - ^(i - ?)<*i*i)- 

v i=2 

Now let 

/9(<rj<ri) 



so that 

JV 

i=2 

Our intention is to apply Lemma 11,41 to show that the second and the 
third terms approximately cancel each other out. First, note that by equa- 
tions (El) and (1261). 



dh ■ P 2 

( 51 ) = -^{-(vj<ri<rk)((Ti) + (<?j<T k ) - (&j<ri)(<Ti<r k )). 

Thus, if we put 

Vk = -<ri<Tk{vi) +Ok~ o-l(crio-fe), and 
w j = -^' CJ i( cr i) + °i ~ (o"iO"i)cri, 

then 

dhj _ (3 2 (<JjVk) _ P 2 (wjVk) 
dgik N ~ N ■ 
From Lemma 13. II it follows that 

^ E ( dhj dh k \ < C(P,h) 
\99ikdgijJ ~ 
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Again, by (jzfl) we get 



A 2 /3 2 ^ F/ . s2.C<J),h) 



3=2 3=2 VJV 

Using the last two bounds in Lemma 11.41 we have 

N N o, v 2 



< £7(0, fc) 



(52) <E-a-E^)' 

Now from (|51|) we see that 

3=2 yiJ 3=2 3=2 



/? 2 " 



iV 

3=2 

Applying ([28]) to the first two terms and ([30]) to the third term, we get 

N q, N 2 



( 53 ) e(E^ + ^( 1 -?)^i> 2 -/ 32 ( 1 -^)) < 
Combining ([52]) and ([53]) . we have 



C(/3,/i) 



iV 2 

E« ??1 > - I) 2 = E(j2g ljhj - /3 2 (1 -,)(!- (a!) 2 )) < ^M. 

3—2 

This proves (|50p and hence completes the proof of Theorem 11.71 
3.6. Proof of Lemma 11.31 For each i,/j£R, let 

p) = cosh(ax + 6)e- (:c -^ 2/2a2 



and 



Let 



r{p) = / u(x)ip b 2{x)dx = — ^roo — -, r-j • 



f(x,p) = I p(t,p)(u(t) - r(p))dt 



p{x,p) 
1 



oo 
oo 



p(x,p) J x 



p(t,p){u(t) - r(p))dt. 



For ease of notation, let us write |^ and |^ as /i and f2- Multiplying 
by p(x,p) on both sides and differentiating w.r.t. x, and finally dividing 
everything by p(x,p), we get 

fx — p \ 
(54) fi(x, p) — ( — 2 atanh(ax + 6) J f(x, p) = u(x) — r(p). 
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This proves the first assertion of the lemma. Now observe that for any x > p, 

' ( + -L h !tj^\ M 

exp I at + b — g — 1 at 

POO 

(55) = e b+a ^ + 2 a ^ 2 / e'^ady. 

J x — fi — aa^ 

Let 



1 2 Z* 00 1 2 

C\(a,a) = a sup e2 z / e~z y dy. 

Z> — CL(7 J z 



It is easy to verify by elementary calculus that C\(a, a) is finite. Since x > p, 
we now get from (I55p that 



exp ( at + o ^— ^ — ] at 



< C 1 (a,a)e^+^ 2 - 2 exp(-^-^l) 

(x-^r 



= C\ (a, cr) exp yax + 6 } 
Similarly, we have 

y exp I —at — o — g — I at < Gi ( — a, <r j exp I —arc — o ^— ^ — I . 

Combining, and putting C2(a,a) = max{Ci(a, a), C\(— a, a)}, we get that 
for x > p, 



(56) / p(t, p)dt < C 2 (a,a)p(x, p). 

J X 

Similarly, if x < p,, we have 

p(t,p)dt < C 3 (a,a)p(x,p) 



for some other constant Cs(a, a). From the definition of /, we can now 
deduce that 

ll/lloo < C 4 (a, ctJHuHoo, 
where C4 = 2max{C2, C3}. Next, note that for x > p, 

( r {t-V) 2 \, 

\t — p\ exp I at + — ^ — lot 



r'' + "" + ^" VJ / "(t-/,)exp(- (t ~^~ 2 Q(j2)2 ^// 



X 

roc 

b+a^+^a 2 / (ay + aa 2 )e~2y 2 ady. 

J x — fi — aiy^ 
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Putting 

1 2 f°° 1 2 

Cs(a, a) = a sup e2 2 / (ay + aa 2 )e~2 y dy = a 2 + aa 2 Ci(a,a), 

z> — aa Jz 

we get 

\t — p\ exp lat + o — ^ — I at 

^ ^ / \ / i (x - m) 2 
< C 5 (a, crjexpl ax + 6 — — 

Proceeding as before, this leads to 

POO 

(57) / \t - p\p(t, p)dt < C 6 (a,a)p(x,p), 



and a similar bound for the integral from — oo to x in the case x < p. An 
easy byproduct of these inequalities is the bound 

(58) sup|(x - p)f(x,p)\ < C 7 (a, cr) |H|oo. 

Using this and (|54l) . and the previous deduction that ||/||oo < C±(a, cr) 1 1 1 1 oo , 
it follows that 

\fi(x,p)\ < C 8 (a, cr) Halloo. 

Now note that 

1 dp x — p 
p dp a 2 

Thus, for x > p, we have 

x-p 1 f°° t-p 

— 2-/(x,/i) p -/ ; 

a 2 p(x,p)J x a 



f 2 (x,p) = —f(x,p) / — —p(t,p)(u(t) - r(p))dt 



oo 



+ 4^1 p(J-P)dl. 



p(x,p) 

Thus from (JSHJ), fl2Z]) and <^ it follows that 

1/2^, P)\ < C 9 (a, a^uWoo + C 2 (a,a)\r'(p)\. 
A simple computation gives 

r , = ^{u^-rjp^p^pY-^dt 
I-00 P) dt 

Thus, 

2|M|oo f°° 

\r'(p)\ < — / \t - p\i> a ,b,^(t)dt. 



From the representation (|13p it follows that 

I* - P\^ a ,b,fi,(r 2 ^ Cio(a, o-)- 

Thus, for x > p, \ f2(x,p)\ < Cn(a,a)\\u\\ 00 . The bound for x < p follows 
similarly. 
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